Machine learning has generally been thought of and attempted to be implemented in the context of artificial intelligence. Artificial intelligence depends on algorithmic solutions (e.g., a computer program) to replicate particular human acts and/or behaviors. The study of neural networks is a sub-area of artificial intelligence which attempts to mimic certain human brain behavior by using individual processing elements that are interconnected by adjustable connections.
In human cognition, perception and understanding of phenomena happen over time and space. That is, we learn things have a common cause based on “temporal adjacency”, meaning that two phenomena around the same time. For example, if a child always sees lightening and then thunder, the child will learn that both phenomena have a common cause. As temporal perception is such an important mechanism in cognition, the inability to model temporal events has been a fundamental limitation of most traditional machine learning models. To this end, Hierarchical Temporary Memories (HTMs) have been developed to simulate temporal aspects of perception and learning. An HTM is a hierarchical network of interconnected nodes that individually and collectively (i) learn, over space and time, one or more causes of sensed input data and (ii) determine, dependent on learned causes, likely causes of novel sensed input data.
One use of HTMs is to learn temporal sequences of sensed input data which have the same cause. These sequences are used in inference to determine causes of new sequences of sensed input data. However, there are instances when sensed inputs with the same cause do not always happen in the same sequence. These circumstances can lead to a large number of similar sequences with the same cause. As learning sequences and their causes is based on the frequency at which the sequences of inputs are sensed, enumerating the frequencies of a large number of sequences can be computationally expensive.
While sequence-based inference is a powerful application of temporal learning, there are also instances when a sequence of sensed inputs is not available and a cause needs to be inferred based on a single sensed input. This is called “flash inference”. While flash inference may be performed using known causes of single inputs alone, this model of learning fails to incorporate the temporal adjacency of inputs with the same cause. Therefore this model is limited by each input being assigned a cause independently of all other inputs over time, even inputs which always occur together or are very similar.